{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Report3 \n",
    "    ·张子龙\n",
    "    ·2018300053\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 问题：服装分类\n",
    "### 1.1任务类型：多分类\n",
    "### 1.2背景介绍：\n",
    "  FashionMNIST 是一个替代 MNIST 手写数字集的图像数据集。 它是由 Zalando（一家德国的时尚科技公司）旗下的研究部门提供。其涵盖了来自 10 种类别的共 7 万个不同商品的正面图片。\n",
    "\n",
    "FashionMNIST 的大小、格式和训练集/测试集划分与原始的 MNIST 完全一致。60000/10000 的训练测试数据划分，28x28 的灰度图片。你可以直接用它来测试你的机器学习和深度学习算法性能，**且不需要改动任何的代码**。\n",
    "\n",
    "这个数据集的样子大致如下（每个类别占三行）："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](fashion-mnist.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 解决思路\n",
    "### 2.1 数据的梳理以及处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "|文件|所含数据类型|内容|\n",
    "|---|---|---|\n",
    "|train-images.npy|array|训练图片集|\n",
    "|train-labels.npy|array|训练图片对应的标签，范围为0~9|\n",
    "|t10k-images.npy|array|测试图片集|\n",
    "|t10k-labels.npy|array|测试图片对应的标签，范围为0~9|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "  看到这个题目的时候，我觉得应该就先把我们拥有的训练集数据来进行训练，然后进行一个多分类的处理，再去完成预测集中的商品的分类。ok，在大概清楚了这些之后我觉得可以开始着手进行报告的编写了。\n",
    "  首先，我们先把文件读取一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "from torch import nn\n",
    "from torch.autograd import Variable\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "#读入Fashionmnist\n",
    "x_train = np.load('train-images.npy')  \n",
    "y_train = np.load('train-labels.npy')\n",
    "x_test = np.load('t10k-images.npy')\n",
    "x_test = np.load('t10k-labels.npy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看一看这几个数据集的大小："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 784)\n",
      "(10000,)\n",
      "(60000, 784)\n",
      "(60000,)\n"
     ]
    }
   ],
   "source": [
    "print(np.shape(x_train))\n",
    "print(np.shape(y_train))\n",
    "print(np.shape(x_test))\n",
    "print(np.shape(y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看的出来每张图片有784个像素点（28* 28），那所以这里对应的tensor尺寸也一定是28* 28。\n",
    "那我们再看看图片大概是什么样子的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAD3CAYAAAA0cknjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAARqUlEQVR4nO3dX4yUZZYG8OfY0NAC0iANNILbA4GNC2aBVNComcxm4+CYSYCLJePFyEYzzIUkOwmJa/RivdlEjcxEkw2RUTK4jg6bMERMJlkmqMFJdGI3gsDADGgYp+0OlDR/RP42ffaiPzYdrO+cor766iv6PL+EdHederveKvuxuuvU+76iqiCi0e2WoidARPlj0IkCYNCJAmDQiQJg0IkCYNCJAhiTZbCIPATgJQAtAF5V1ees60+bNk27urqy3GQ4Fy9eNOtffPGFWZ8yZUpq7dZbbzXHikjNdW/ep06dMuvjxo0z6zNnzkyttbS0mGNHq56enq9UtaNSreagi0gLgP8C8CCAXgAfi8gOVf1T2piuri50d3fXepO5st5P4P3A5+nQoUNmfd26dWZ99erVqbUlS5aYY1tbW836mDHpPz4HDx40x27fvt2sz50716w/+eSTqbX29nZz7GglIn9Nq2X51X0ZgKOq+rmqXgbwGwArMnw/IspJlqDfAeBvI77uTS4joiaTJeiVfp/91u+/IrJWRLpFpLtcLme4OSKqVZag9wKYM+Lr2QD6rr+Sqm5S1ZKqljo6Kr5OQEQ5yxL0jwHMF5HviEgrgB8B2FGfaRFRPdX8qruqDorIOgD/i+H22mZVtV9qJaJCSCOXqZZKJc2rvebdjzxbZJ988olZ37p1a2pt27Zt5livJ3zu3DmzfuHChdTawMCAOTZPCxYsMOu33GL/snn48OHUmtVjB4Dly5eb9fXr16fW7r77bnNskUSkR1VLlWp8ZxxRAAw6UQAMOlEADDpRAAw6UQAMOlEAo6a9ltXZs2dTa48++qg5dt++fWbdeownTpxojm1razPr1goywG7PDQ4OmmPPnDlj1q1lrl5bMM92p7dE1mo5AsDly5dTaw888IA59o033jDreWJ7jSg4Bp0oAAadKAAGnSgABp0oAAadKAAGnSiATNs9jyarVq1KrXlbKs+YMcOsWz3jq1evmmOzbl1sfX/vPRS33357zd/bk+f7N7z3HowfP96sW/+9PvjgA3Ost2vvXXfdZdbzwmd0ogAYdKIAGHSiABh0ogAYdKIAGHSiABh0ogDC9NF7enrMutUrnzZtmjnWW9dt8dZGf/nll5nGDw0Npda8texen9zbktlirfkGgLFjx5r1SZMmpdZmz55tjvXut8W7z6+++qpZ37BhQ823nQWf0YkCYNCJAmDQiQJg0IkCYNCJAmDQiQJg0IkCyNRHF5FjAL4GcBXAYNqe0s3gvffeM+uXLl1KrXn7hHu9VauXPW7cOHPsCy+8YNY7OzvN+pw5c1JrfX19mb63db+8PrjXR/eOg96zZ09q7eWXXzbHdnR0mPUrV66k1rz/1t4x2EX10evxhpl/UtWv6vB9iCgn/NWdKICsQVcAO0WkR0TWVrqCiKwVkW4R6S6XyxlvjohqkTXo96vqUgA/APCEiHz3+iuo6iZVLalqyfvbiIjykSnoqtqXfDwBYDuAZfWYFBHVV81BF5EJIjLp2ucAvg/gQL0mRkT1U/OxySIyF8PP4sDwq/dvqup/WmOKPDb53nvvNevHjx9Prd12223m2NbWVrNutYomT55sjv3oo4/M+s6dO816b29vau2xxx4zx77yyitmfeHChak1ryXpLYGdPn26WV+yZElqbf78+eZY76hqa+5e2/Dw4cNm/cAB+7lwwYIFZt1iHZtcc3tNVT8H8I81z4qIGobtNaIAGHSiABh0ogAYdKIAGHSiABh0ogDCbPe8b98+s24t5/R6vtYSV8+ZM2dqHgsAy5cvN+tWz9g74vfFF18069ZR0++884451tsi2+qTA/YyVW875/Pnz5t1aymqt0zV+jkCgA8//NCsZ+mjW/iMThQAg04UAINOFACDThQAg04UAINOFACDThTAqOmj79+/36x721i1tLSk1rw+ule3jjaeOnWqOdZz8OBBs25tJ93f32+OfeaZZ8y6tZeBt27b2wfB6zdbvG2qvW2urZ8FETHHtrW1mfXdu3eb9TVr1pj1WvEZnSgABp0oAAadKAAGnSgABp0oAAadKIBR0157/vnnzbrV4gKACRMmpNayLnscP358as1rQ3nbY588edKsDwwMpNasU0MBewtswJ67dZ8B/zTV06dPm/WtW7em1k6dOmWO9Vpg1m17Y73HtKenx6znhc/oRAEw6EQBMOhEATDoRAEw6EQBMOhEATDoRAG4fXQR2QzghwBOqOqi5LKpALYC6AJwDMBqVbWblzm77777zLrXEz569GhqzduS2eujW8f4etsH33PPPWbdWlLpfX/vtoeGhsy61TP2lqF6703wlv5aR1l7WyZ/8803Zt263979mjVrlllfuXKlWc9LNc/ovwLw0HWXPQVgl6rOB7Ar+ZqImpQbdFXdDeD6t1etALAl+XwLgJX1nRYR1VOtf6PPUNV+AEg+Tq/flIio3nJ/MU5E1opIt4h0l8vlvG+OiCqoNejHRaQTAJKPJ9KuqKqbVLWkqiVv3zYiyketQd8B4NoudmsAvF2f6RBRHtygi8hbAD4E8Pci0isijwN4DsCDInIEwIPJ10TUpMTrC9ZTqVRSb311Uaw1zEeOHDHHbty40ay///77qbU777zTHOv18Nvb2826te7b65Pnyfu58+ZmrXf3HrNFixaZ9TfffNOsNysR6VHVUqUa3xlHFACDThQAg04UAINOFACDThQAg04UAINOFMCo2dc9qylTpqTWli1bZo61jiYGgHfffTe15h3De+nSJbPura0eHBxMrXnr0T1WL9zrk3u37d1va0/5ixcvmmO9vQtGIz6jEwXAoBMFwKATBcCgEwXAoBMFwKATBRCmvea1e6yti1tbW82xXots0qRJqTVvW2NvO2fvti3eY5Lle+ctyxJbb2mvxfvv5bUNi3pM+YxOFACDThQAg04UAINOFACDThQAg04UAINOFECYPrrXv7SWPXrmzZtn1q0jfq1lpIDfw/dY97uZ++je/ba2sfZMnjy55rFe/95730NR+IxOFACDThQAg04UAINOFACDThQAg04UAINOFIDbRxeRzQB+COCEqi5KLnsWwE8AlJOrPa2qv8trko1g9Ue93mhbW5tZt7aD9rYm9vr71jp6wO6Ve31yr89u1bMeyWwdiwwA58+fT615827WXneeqnlG/xWAhypc/gtVXZz8u6lDTjTauUFX1d0ABhowFyLKSZa/0deJyKcisllEUo85EZG1ItItIt3lcjntakSUo1qDvhHAPACLAfQD2JB2RVXdpKolVS11dHTUeHNElEVNQVfV46p6VVWHAPwSgH04GREVqqagi0jniC9XAThQn+kQUR6qaa+9BeB7AKaJSC+A/wDwPRFZDEABHAPw0/ym2BhZlmR6W/xade92s7bAsnzvLC2yLK25aliPadYtmS3NvAW2xQ26qj5S4eLXcpgLEeWE74wjCoBBJwqAQScKgEEnCoBBJwqAQScKIMx2z0Xq6+tLrXlH+Ho9YY/Vr87ay86TNzdr+a431ttiezTiMzpRAAw6UQAMOlEADDpRAAw6UQAMOlEADDpRAOyjJ/JcZ5xle2HveGBvbXWWPnqWY5WzHsns3W9rC23ve2fpo9+s69H5jE4UAINOFACDThQAg04UAINOFACDThQAg04UAPvoDWD1fL29070evDc+y57yXr/ZWhPufW9vnb03fsyY2n90T58+XfPYmxWf0YkCYNCJAmDQiQJg0IkCYNCJAmDQiQKo5tjkOQBeBzATwBCATar6kohMBbAVQBeGj05eraqn8pvqzSvLMlVP1uWgFq8FlqXFlfU4aGu81fYDgAsXLpj1Wm+3mVXzjD4IYL2q3gXgXgBPiMg/AHgKwC5VnQ9gV/I1ETUhN+iq2q+qe5LPvwZwCMAdAFYA2JJcbQuAlTnNkYgyuqG/0UWkC8ASAH8EMENV+4Hh/xkAmF732RFRXVQddBGZCGAbgJ+p6tkbGLdWRLpFpLtcLtcyRyLKqKqgi8hYDIf816r62+Ti4yLSmdQ7AZyoNFZVN6lqSVVLHR0d9ZgzEd0gN+gy/DLjawAOqerPR5R2AFiTfL4GwNv1nx4R1UM1/ZH7AfwYwH4R2Ztc9jSA5wD8j4g8DuALAP+SywyJKDM36Kr6BwBpzcN/ru90RqesRx9b8uzr5nmssjdvb/mtNd5738L58+fN+mjEd8YRBcCgEwXAoBMFwKATBcCgEwXAoBMFwKATBcDtnhNFrTP2+sV5ytonz/L+gKzr0a3HzVsnn+f7GpoVn9GJAmDQiQJg0IkCYNCJAmDQiQJg0IkCYHstYbVzsrbeWltbU2tZth6uhnWaap4nuVq3W40s7Tdv3nm2BZsVn9GJAmDQiQJg0IkCYNCJAmDQiQJg0IkCYNCJAmAfvWBZe9lZlnN6t+3VrV551iWwWbeDtnCZKhGNSgw6UQAMOlEADDpRAAw6UQAMOlEADDpRAG4fXUTmAHgdwEwAQwA2qepLIvIsgJ8AKCdXfVpVf5fXRPOW5zrjWbNmpdaOHDlijvW2LvbWfVv1y5cvZ/re1mPmPZ7e/bpy5YpZzyLievRq3jAzCGC9qu4RkUkAekTk90ntF6r6Yn7TI6J6cIOuqv0A+pPPvxaRQwDuyHtiRFQ/N/Q3uoh0AVgC4I/JRetE5FMR2SwiU+o9OSKqj6qDLiITAWwD8DNVPQtgI4B5ABZj+Bl/Q8q4tSLSLSLd5XK50lWIKGdVBV1ExmI45L9W1d8CgKoeV9WrqjoE4JcAllUaq6qbVLWkqqWOjo56zZuIboAbdBl+mfE1AIdU9ecjLu8ccbVVAA7Uf3pEVA/VvOp+P4AfA9gvInuTy54G8IiILAagAI4B+GkO8yOiOqjmVfc/AKjUPLxpe+aNdvr06dTauXPnzLFeP/nkyZNm3eoZe2u68+xlZz3aePbs2ak1b6/8zz77zKxbsqzRL1JzzoqI6opBJwqAQScKgEEnCoBBJwqAQScKgNs9J/I8Nnnp0qWptYULF5pj29vbzXqWFpjXKpo4caJZtx4Xb7vnrMtvx44dm1qz2pkAsGxZxTdxVqVZ22eem3PWRHRDGHSiABh0ogAYdKIAGHSiABh0ogAYdKIAJOvxtjd0YyJlAH8dcdE0AF81bALVa9Z5AZxbLZp1XkB95/Z3qlpxG6eGBv1bNy7SraqlwiaQolnnBXButWjWeQGNmxt/dScKgEEnCqDooG8q+PbTNOu8AM6tFs06L6BBcyv0b3Qiaoyin9GJqAEKCbqIPCQifxaRoyLyVBFzSCMix0Rkv4jsFZHugueyWUROiMiBEZdNFZHfi8iR5GPDj8JKmdezIvJl8rjtFZGHC5jXHBF5T0QOichBEfm35PJmeMzS5taQx63hv7qLSAuAvwB4EEAvgI8BPKKqf2roRFKIyDEAJVUtvO8qIt8FcA7A66q6KLnsBQADqvpc8j/JKar6700wr2cBnCvydN3kUJHOkSf/AlgJ4F9R/GOWNrfVaMDjVsQz+jIAR1X1c1W9DOA3AFYUMI+mp6q7AQxcd/EKAFuSz7dg+IeloVLmVThV7VfVPcnnXwO4dvJvMzxmaXNriCKCfgeAv434uhfNdQyzAtgpIj0isrboyVQwIznK+tqR1tMLns9ITXO67nUn/zbVY1bEqcRFBL3S/kPN9NL//aq6FMAPADyR/JpKvqpO122ECif/No1aTyXOqoig9wKYM+Lr2QD6CphHRaral3w8AWA7Uk6JLdDxawdcJh9PFDwfANWfrpu3Sif/okkesyynEmdVRNA/BjBfRL4jIq0AfgRgRwHz+BYRmZC8UAIRmQDg+2i+U2J3AFiTfL4GwNsFzuX/NcPpumkn/6IJHrPCTyVW1Yb/A/Awhl95/wzAM0XMIWVecwHsS/4dLHpuAN7C8K9zVzD8m9DjAG4HsAvAkeTj1CaZ138D2A/gUwwHq7OAeT2A4T8DPwWwN/n3cJM8Zmlza8jjxnfGEQXAd8YRBcCgEwXAoBMFwKATBcCgEwXAoBMFwKATBcCgEwXwf6OKIi9NfK7TAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "image = x_test[1].reshape(28,28)\n",
    "fig = plt.figure(figsize=(3, 3))  \n",
    "fig.subplots_adjust(left=0, right=1, bottom=0, top=1, hspace=0.05, wspace=0.05)\n",
    "ax = fig.add_subplot()\n",
    "ax.imshow(image, cmap=plt.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了方便，这里我选择了多层感知机模型来进行数据的处理，以 728 个点的像素作为输入，为防止信息损失过快，设定了(400，200，100)的隐藏层。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MLPClassifier(alpha=1e-05, hidden_layer_sizes=(400, 200, 100), random_state=1)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.neural_network import MLPClassifier\n",
    "mlp = MLPClassifier(solver='adam', alpha=1e-5,hidden_layer_sizes=(400,200,100), random_state=1, activation='relu')\n",
    "mlp.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再来计算PR-AUC："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "acc_train = 0.926300,acc_test=0.821117\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import average_precision_score\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "y_train_pred = mlp.predict(x_train)\n",
    "y_test_pred = mlp.predict(x_test)\n",
    "#PR-AUC\n",
    "#计算准确度\n",
    "acc_train = accuracy_score(y_train, y_train_pred)\n",
    "acc_test = accuracy_score(y_test,y_test_pred)\n",
    "print(\"acc_train = %f,acc_test=%f\" % (acc_train,acc_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然方法选择的比较简单，但是精度还是在可以接受的范围之内。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 爬虫的学习以及应用\n",
    "  其实一开始并不是很了解爬虫的作用之类，所以在网上查找了想关的资料并进行了简单的学习。csdn上的这个帖子[Python爬虫入门项目](https://blog.csdn.net/u014044812/article/details/78894108)感觉起来还不错，我也是看这个帖子简略的了解了一下爬虫。\n",
    "  开始爬取所需的内容："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\bs4\\__init__.py:221: UserWarning: You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored.\n",
      "  warnings.warn(\"You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored.\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[<img alt=\"新品 户外双肩背包书包电脑包BA5927 078 NIKE耐克男包女包2020秋季\" src=\"https://img.alicdn.com/bao/uploaded/i1/3465928182/O1CN01gdYKfK2AJPEk10ZBE_!!3465928182.jpg_310x310.jpg\"/>, <img alt=\"小椰子回到未来黑白奥利奥潮流跑步鞋 简版 002 844839 Kwazi Nike\" src=\"https://img.alicdn.com/bao/uploaded/i1/3332387846/O1CN01sVImu427pWB4TdTQG_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"M2K耐克黑红白蓝色复古运动跑步鞋 Monarch 415445 老爹鞋 Air Nike\" src=\"https://img.alicdn.com/bao/uploaded/i2/789110989/O1CN01qzrPnX1JB0HOwLP81_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"101 IV老爹鞋 415445 M2K黑红白蓝慢跑鞋 102 MONARCH Air 耐克Nike\" src=\"https://img.alicdn.com/bao/uploaded/i1/87912102/O1CNA1Kbvxnh1ROl9FpYxFV_!!87912102-0-psf.jpg_310x310.jpg\"/>, <img alt=\"休闲跑步鞋 新款 轻便运动鞋 男鞋 男 2020秋冬季 男官网旗舰正品 耐克鞋\" src=\"https://img.alicdn.com/bao/uploaded/i3/3190386149/O1CN01EWknt11vII3Ur1cOb_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"AQ2203 男女全掌气垫跑步鞋 飞马36新款 002 Pegasus Zoom Nike\" src=\"https://img.alicdn.com/bao/uploaded/i2/3332387846/O1CN01DNCHSl27pWB1kTOJn_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"905237 916274 010 男子加绒针织收口休闲直筒长裤 NIKE耐克\" src=\"https://img.alicdn.com/bao/uploaded/i2/763026234/O1CN01E4hyXT1vvDe3dL31O_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"5夏季 伦敦三代跑步鞋 女6六 网面透气学生情侣运动鞋 爱耐克男鞋 正品\" src=\"https://img.alicdn.com/bao/uploaded/i2/4184133481/O1CN01hTyJHo1baLIJWBAcp_!!4184133481.jpg_310x310.jpg\"/>, <img alt=\"FLEX赤足轻便透气运动跑步鞋 013 AJ5900 19情侣新款 NIKE耐克男女\" src=\"https://img.alicdn.com/bao/uploaded/i1/53725805/O1CN01dV4cfk1skjnacMaD1_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"315122 板鞋 AF1纯白空军一号男女休闲鞋 111 Force Air 耐克Nike\" src=\"https://img.alicdn.com/bao/uploaded/i1/735464980/O1CN01Uik4G01mesxi39kKm_!!735464980.jpg_310x310.jpg\"/>, <img alt=\"Pegasus飞马35男女缓震全掌气垫跑步鞋 Zoom 001 942851 Air Nike\" src=\"https://img.alicdn.com/bao/uploaded/i1/3332387846/O1CN019cmXQj27pWB6PXPel_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"潮流缓震运动跑鞋 TL男子复古全掌气柱老爹鞋 002 AV3595 Shox Nike\" src=\"https://img.alicdn.com/bao/uploaded/i4/3332387846/O1CN01jnosnT27pWB95Sthw_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"DA4086 Mid 黑白开拓者中帮男女休闲板鞋 100 Blazer 耐克 Nike\" src=\"https://img.alicdn.com/bao/uploaded/i1/735464980/O1CN01qrrggA1met2pQ0GvC_!!735464980.jpg_310x310.jpg\"/>, <img alt=\"GS3901 010 CU1589 耐克加绒保暖防滑触屏足球运动手套 天天正品\" src=\"https://img.alicdn.com/bao/uploaded/i1/1055775210/O1CN01arARWH1oME8SLF6wb_!!1055775210.jpg_310x310.jpg\"/>, <img alt=\"002 AQ2235 001 CQ7628 MAX气垫缓震透气运动跑步鞋 Nike耐克男AIR\" src=\"https://img.alicdn.com/bao/uploaded/i1/53725805/O1CN01a9IYHE1skjnSP6R4w_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"React DC0833 气垫电玩彩虹男女子运动休闲跑步鞋 AirMax270 Nike\" src=\"https://img.alicdn.com/bao/uploaded/i2/3332387846/O1CN01TS7maU27pWB95UJ2S_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"KD杜兰特星空蓝球AJ乔丹 耐克NIKE篮球室内外7号比赛詹姆斯限量版\" src=\"https://img.alicdn.com/bao/uploaded/i1/2204174961912/O1CN01IaLzQm1PzjvdC9m9f_!!2204174961912-0-picasso.jpg_310x310.jpg\"/>, <img alt=\"BQ5832 011 100 Snood运动面罩耳罩围脖三合一CT3103 耐克Strike\" src=\"https://img.alicdn.com/bao/uploaded/i4/35454613/O1CN01gJTf9x1jwnhbeDViI_!!35454613.jpg_310x310.jpg\"/>, <img alt=\"女子大气垫运动休闲缓震跑步鞋 VAPORMAX AJ6910 AJ6900 AIR Nike\" src=\"https://img.alicdn.com/bao/uploaded/i1/59290313/O1CN013oAEwK1EBOboPHgxm_!!59290313.jpg_310x310.jpg\"/>, <img alt=\"010 FORCE男子纯黑白潮流滑板鞋 942237 休闲鞋 100 DELTA NIKE\" src=\"https://img.alicdn.com/bao/uploaded/i1/3332387846/O1CN01QPwzox27pWB42aFjN_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"男女2020新款 三双装 运动中高帮长筒跑步透气潮流袜子 NIKE耐克正品\" src=\"https://img.alicdn.com/bao/uploaded/i4/716333938/O1CN01Acm3di1exeJIJoneE_!!716333938-0-lubanu-s.jpg_310x310.jpg\"/>, <img alt=\"男女实战篮球鞋 Air Off AR6346 AJ6简版 AR4430 Lift Jordan Nike\" src=\"https://img.alicdn.com/bao/uploaded/i4/3332387846/O1CN01eyc2Un27pWAvqjwYg_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"男子运动训练飞线缓震轻质跑步鞋 SWIFT 013 908989 RUN Nike\" src=\"https://img.alicdn.com/bao/uploaded/i1/59290313/O1CN01h4KWee1EBOZcMQxeE_!!59290313.jpg_310x310.jpg\"/>, <img alt=\"男女袜子运动高中低筒中帮短袜长袜三双装 SX7676 Nike耐克2020四季\" src=\"https://img.alicdn.com/bao/uploaded/i2/3911453906/O1CN01jhA9OZ1eizdeWOlOg_!!3911453906.jpg_310x310.jpg\"/>, <img alt=\"男女户外双肩包校园书包大容量背包BA6170 077 Nike耐克2020年新款\" src=\"https://img.alicdn.com/bao/uploaded/i1/3465928182/O1CN01PPpWqn2AJPCi5amH3_!!3465928182.jpg_310x310.jpg\"/>, <img alt=\"王一博同款 CN5433 LOGO运动休闲梭织男子束脚滑板长裤 耐克Nike\" src=\"https://img.alicdn.com/bao/uploaded/i4/3332387846/O1CN01DAUkS327pWAvqhGE7_!!0-item_pic.jpg_310x310.jpg\"/>, <img alt=\"毛巾底中筒吸汗运动中帮袜子 7667 SX7677 三双装 Nike耐克男女新款\" src=\"https://img.alicdn.com/bao/uploaded/i4/716333938/O1CN01eSzArg1exeJ3nPilF_!!716333938-0-lubanu-s.jpg_310x310.jpg\"/>, <img alt=\"BQ7474 013 010 保暖卫衣男半拉链套头衫 扎吉体育Nike足球运动冬季\" src=\"https://img.alicdn.com/bao/uploaded/i1/35454613/O1CN01PD840b1jwnhWjSJIB_!!35454613.jpg_310x310.jpg\"/>, <img alt=\"CK2956 泡棉机能缓震男跑步鞋 700 AV2605 601 Presto React Nike\" src=\"https://img.alicdn.com/bao/uploaded/i1/1607340948/O1CN01zkx1P31IsE5umDUWq_!!1607340948.jpg_310x310.jpg\"/>, <img alt=\"AO3108 500 运动休闲鞋 BQ3378 潮鞋 女子运动M2K老爹鞋 Nike耐克\" src=\"https://img.alicdn.com/bao/uploaded/i1/762010217/O1CN01Kyq44p1DTQZuTHGuj_!!762010217.jpg_310x310.jpg\"/>]\n",
      "image/0.jpg\n",
      "image/1.jpg\n",
      "image/2.jpg\n",
      "image/3.jpg\n",
      "image/4.jpg\n",
      "image/5.jpg\n",
      "image/6.jpg\n",
      "image/7.jpg\n",
      "image/8.jpg\n",
      "image/9.jpg\n",
      "image/10.jpg\n",
      "image/11.jpg\n",
      "image/12.jpg\n",
      "image/13.jpg\n",
      "image/14.jpg\n",
      "image/15.jpg\n",
      "image/16.jpg\n",
      "image/17.jpg\n",
      "image/18.jpg\n",
      "image/19.jpg\n",
      "image/20.jpg\n",
      "image/21.jpg\n",
      "image/22.jpg\n",
      "image/23.jpg\n",
      "image/24.jpg\n",
      "image/25.jpg\n",
      "image/26.jpg\n",
      "image/27.jpg\n",
      "image/28.jpg\n",
      "image/29.jpg\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "import requests\n",
    "import os\n",
    "from bs4 import BeautifulSoup\n",
    " \n",
    "url = 'https://orz123.cn/search/3sLcrOD9.html'\n",
    "html = requests.get(url).text  #获取网页内容\n",
    "\n",
    "soup = BeautifulSoup(html,'html.parser',from_encoding='utf-8')\n",
    "\n",
    "pic_url = soup.find_all('img',src=re.compile(r'^https://img.alicdn.com/bao/uploaded/.*?jpg$'))\n",
    "\n",
    "print(pic_url)\n",
    "i = 0\n",
    "#判断image文件夹是否存在，不存在则创建\n",
    "if not os.path.exists('image'):\n",
    "    os.makedirs('image')\n",
    "for url in pic_url:\n",
    "    img = url['src']\n",
    "    try:\n",
    "        pic = requests.get(img,timeout=5)  #超时异常判断 5秒超时\n",
    "    except requests.exceptions.ConnectionError:\n",
    "        print('当前图片无法下载')\n",
    "        continue\n",
    "    file_name = \"image/\"+str(i)+\".jpg\" #拼接图片名\n",
    "    print(file_name)\n",
    "    #将图片存入本地\n",
    "    fp = open(file_name,'wb')\n",
    "    fp.write(pic.content) #写入图片\n",
    "    fp.close()\n",
    "    i+=1\n",
    "#https://img.alicdn.com/bao/uploaded/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 处理爬虫得到的图片\n",
    "  利用上课的时候老师课件里用到的方法来对图片进行处理，以便我们后续工作的进行。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "先将图片进行灰度化处理："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "before max pool, image shape: 310 x 310\n",
      "after max pool, image shape: 28 x 28 \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1eb007e9970>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVlklEQVR4nO3da2yVZbYH8P9qRYFyKQWKtCCCXCKcRDQFb2g8mThBvniJc6IxE09iZDRDMpPMh2M0cfwmHp0Z58PJJJ2jjmM8GpMZghqdM0bHoEQHCnJTDqJQroUCBWxpq1DW+dBtUrHvf3X22+69M8//l5C2e/XZ++m792Jf1ruex9wdIvLPr6rcExCR0lCyiyRCyS6SCCW7SCKU7CKJuKiUN2ZmXlWV/f8Li0XxUaNG0bHV1dU0HlUl+vr6MmPnzp2jY8eOHUvjp06dovFo7mZWVAyI/+5LLrmExqP7LE+15/z58zQeHfc81503HhkzZsyIXPfZs2fR19c36J2eK9nNbDmA3wKoBvDf7r6a/X5VVRVGjx6dGR83bhy9PZY0DQ0NdGx03SyZAZ6Qx48fp2ObmppofM2aNTQ+fvx4GmfHNErG6IE1a9YsGo+O69mzZ4u+7d7eXhpvb2+ncfa3d3V10bHd3d003tPTQ+PRf3Lz58/PjEVzY//579u3LzNW9Mt4M6sG8F8AbgOwEMC9Zraw2OsTkZGV5z37UgBfuPsed/8GwKsAbh+eaYnIcMuT7I0ADgz4+WDhsu8ws5Vm1mJmLTpbT6R88rxnH+xDgO9ls7s3A2gGgOrqamW7SJnkeWY/CGDmgJ9nADicbzoiMlLyJPtGAPPMbLaZXQzgHgCvD8+0RGS4Ff0y3t3PmdkqAP+L/tLb8+7+aTQuqvsyrJTS0dFBx0a17gkTJtA4q4tGf9OJEydoPBof1dlZPBob1fhPnz5N46zsB/Cy4d69e+nYRYsW0fiRI0do/KKLsh/eUdkv7/kD0X3KjltnZycdG5WJs+Sqs7v7WwDeynMdIlIaOl1WJBFKdpFEKNlFEqFkF0mEkl0kEUp2kUSUtJ89EvUns9poNDZqQ83TRlpbW0vHRrXuqGc86tVn9eTotqM6eTQ+mhtrgY3GtrW10XiePv9obFTLjuroUZydv5DnHABW/9czu0gilOwiiVCyiyRCyS6SCCW7SCKU7CKJqKjSW55WzqjlMGqBZat9AkB9fX1mbNKkSXTs0aNHaXz27Nk0fvLkSRpnolbNqOQYie4z1lq8bNkyOjYqQUWlOVaSZDEgf4trFL/tttsyY2vXrqVji6VndpFEKNlFEqFkF0mEkl0kEUp2kUQo2UUSoWQXSUTJ6+zFtucBwNdff50Ze/vtt4ueExDXVZ999tnMWF1dHR0b1YvzLufMasZRC2t025dddhmNR3V6dv3R/c2W7wbiFlkmOj/gzTffpPGHH3441/Wzx0ze7aCz6JldJBFKdpFEKNlFEqFkF0mEkl0kEUp2kUQo2UUSUfI6e1RbLVZUT547d26u6//oo48yY01NTXTshx9+SON5esIBvox2tMR2T08PjbNzG4B4yWVWC49uO5p7ZMaMGZmxAwcO0LF5H6fR+EOHDo3YbWfJlexm1gqgE0AfgHPuzh/1IlI2w/HM/q/uzndgEJGy03t2kUTkTXYH8Fcz22RmKwf7BTNbaWYtZtYyUu9FRCSW92X8je5+2MzqAbxjZv/n7usG/oK7NwNoBoDq6mplu0iZ5Hpmd/fDha/tANYAWDockxKR4Vd0sptZjZmN//Z7AD8EsGO4JiYiwyvPy/hpANYUtqa9CMD/uPtf8kwmqtlefPHFmbFoS+ZobfZdu3bReHd3d2Ys2p737rvvpvHm5mYaZ9seA7yWHdWq82y5DAATJkygcdZrH902u7+BeKvrzs5OGmc2bNhQ9NihYPsURGszFPvZV9HJ7u57AFxV7HgRKS2V3kQSoWQXSYSSXSQRSnaRRCjZRRJR8hZXtkxuVHo7e/ZsZmzVqlV0bFQei7A20927d9OxjY2NND5nzhwab29vp3F23KLyVHRcurq6aDwqvUXtu3mwxwPAy37R2NWrV9N4dFwi77//fmYsOmZRnmTRM7tIIpTsIolQsoskQskukgglu0gilOwiiVCyiySipHX2qqoqWq+OWvdY/TFa8njy5Ml8cgF221HNddOmTTS+ZMkSGn/jjTdonNXKoxbX3t5eGv/mm29o/OTJkzQ+ZcqUzNiZM2fo2Ki9NmqBZcclGhuJauHRtsufffZZZoydHwCozi4iASW7SCKU7CKJULKLJELJLpIIJbtIIpTsIokoaZ3dzOjywWx5XQBoa2vLjEU12WhL52hZ49ra2sxYVBfN20s/kj3hY8aMofGqKv58EJ3fwM5BiO6zaKvqq67iixsfOXIkM3bq1Ck6Nu99Fo1n6wxEx3Tq1KmZsWPHjmXG9Mwukgglu0gilOwiiVCyiyRCyS6SCCW7SCKU7CKJKGmd/dy5c7S+uXHjRjq+oaEhMxbVuidOnEjjUV83q/lGa7Pn3Zp4wYIFNL5nz57MWHRcTp8+TePR3KJzAMaPH58Zq6mpoWMjrI4OAHfeeWdm7Omnn6Zje3p6aDw6/yCKs7UbRuq8ivCZ3cyeN7N2M9sx4LI6M3vHzHYXvk4akdmJyLAZysv4PwBYfsFljwB4193nAXi38LOIVLAw2d19HYCOCy6+HcCLhe9fBHDH8E5LRIZbse/Zp7l7GwC4e5uZ1Wf9opmtBLCy8H2RNycieY34B3Tu3gygGQCqqqr4ipIiMmKKLb0dNbPpAFD4yrcZFZGyKzbZXwdwf+H7+wGsHZ7piMhICV/Gm9krAG4BMMXMDgL4JYDVAF4zswcA7Afwo6HcmJnR+mO0bjxbAz1a3zyqm0b7jLNaeVSLjurJV155JY0/8ggvdlx//fU0zkR1+GhdeNZbDfD7LKqTR2sQvPTSSzTOzul45pln6NjouOTF1hGI1tM/evRoZoztOx/+Re5+b0boB9FYEakcOl1WJBFKdpFEKNlFEqFkF0mEkl0kESVfSjpPSYOVuKKWwqg0F22Dy1o1ly+/sE/ouxYtWkTjM2bMoPEI2/73tddeo2NZGQeIt4uOjjtbSnru3Ll07K233krjkcbGxszYhg0b6NiFCxfSeNQSHZVjb7755szYunXr6FiWQ1pKWkSU7CKpULKLJELJLpIIJbtIIpTsIolQsoskoqR19vr6ejz44IOZ8bq6OjqeLbHb2tpKx7JlqIF4ySzWdhgtU11fn7lq15BES1Gz1uD77ruPjo3OP1iyZAmNr1+/nsbZNtzLli2jY6PHA2ufjUTH9NChQzTOavhA3DI9aVL2gszRls2sxs8eC3pmF0mEkl0kEUp2kUQo2UUSoWQXSYSSXSQRSnaRRFi0fPNwmjhxot9www2ZcbbFLsD7eKMlj6dMmULjS5cupXFWp496l6MlkTs7O2k8WgOAnX8Q9elHovMP2NLFAO93P3/+PB0b3adsG22Ab6Ud9eFH1x3V+O+55x4aP3z4cGaMrQEA8Pu0tbUVPT09g95pemYXSYSSXSQRSnaRRCjZRRKhZBdJhJJdJBFKdpFElLSf/cyZM/j4448z41Gt+7rrrsuMsfWyAeDaa6+l8draWhpn9WRW5waA/fv30/jkyZNpPKpHszp8dA5AVC+O1kdna9YDvG88um22Vj8Qb8MdbSfNRHOLzp144YUXaHzFihWZMbadM8D3Ici1bryZPW9m7Wa2Y8BlT5jZITPbUviXPXMRqQhDeRn/BwCDbXnyG3dfXPj31vBOS0SGW5js7r4OQEcJ5iIiIyjPB3SrzGxb4WV+5oJaZrbSzFrMrCV67ykiI6fYZP8dgCsALAbQBuBXWb/o7s3u3uTuTVHzgYiMnKKyz92Punufu58H8HsA/GN0ESm7opLdzKYP+PFOADuyfldEKkNYZzezVwDcAmCKmR0E8EsAt5jZYgAOoBXAT4Z6g6wmvXjxYjqW9TdH+2lHtcuODv4ZJKsXRz3f0RriUU/4uHHjaDzP+unR5yhRrfvqq6+mcda33d3dTcdGNf48azFE90kUj44566UHgPfeey8zFq3lzx5v27Zty4yFye7u9w5y8XPROBGpLPrETCQRSnaRRCjZRRKhZBdJhJJdJBElbXEFeKknalNdvnywfpx+u3fvpmOj5Zqjs/tYqSUq60Vb8EaiEhQTLUMdtcBGc4/+9tmzZ2fGDhw4kOu2o/uMHbeo1Bpdd1SyjJaiZtcflRRZaY7NS8/sIolQsoskQskukgglu0gilOwiiVCyiyRCyS6SiJLX2Vl7XtSmylpBr7nmGjo2Wna4vb2dxlmrZ1Srjmqy0VLUedopo5ptFI+WTM7TZlpfX0/jUetwVIc/dOhQZmzGjBl0bFRnj+6zqG2Znf8QXTe7v9lYPbOLJELJLpIIJbtIIpTsIolQsoskQskukgglu0giSlpnnzZtGh566KHMeNQbnWdHmaivu66ujsbZMtYnTpygY7dv307jV1xxBY1HyzWzOnz0d0d18rxbdrG6b1TDj2rVUc/4/PnzM2Nbt26lY6NzJ6JlsKNzCCZOnJgZ++STT+hYdv5BV1dXZkzP7CKJULKLJELJLpIIJbtIIpTsIolQsoskQskukoiS1tnHjBmDRYsWZcajtd1Z7TPaWngke84nT55Mx86bN4/GWT0YAPr6+micbScd/V1Rz3i0fXBUx2dzj2r80XkVUR2enX+wYMECOvbgwYM0vn79ehrfvHkzjbM17aM8YNtg79+/PzMWPrOb2Uwz+5uZ7TSzT83sZ4XL68zsHTPbXfg6KbouESmfobyMPwfgF+5+JYDrAPzUzBYCeATAu+4+D8C7hZ9FpEKFye7ube6+ufB9J4CdABoB3A7gxcKvvQjgjhGao4gMg3/oAzozuxzA1QD+DmCau7cB/f8hABj0ZGAzW2lmLWbWcvr06ZzTFZFiDTnZzWwcgD8B+Lm7fzXUce7e7O5N7t7ETv4XkZE1pGQ3s1HoT/SX3f3PhYuPmtn0Qnw6AL48q4iUVVh6s/7azHMAdrr7rweEXgdwP4DVha9ro+vq7e3Frl27MuN33XUXHV9TU5MZi0prrPUPiEtUrIwTlZAaGxtpPM/2vgBfBnvq1Kl0bPTWKvrb8ixzHZXOpk+fTuOs5AjwcumWLVvo2H379tE4K3EBwJdffknjeR5PtbW1mTFWCh1Knf1GAD8GsN3MthQuexT9Sf6amT0AYD+AHw3hukSkTMJkd/cPAWSdefGD4Z2OiIwUnS4rkgglu0gilOwiiVCyiyRCyS6SiJK2uI4dO5Yui9zW1kbHs/pjtH3v3LlzaTyq+e7duzczdtNNN9GxURtodNuRhoaGzFi0/W+0fPdILzXNRO23UevvgQMHMmNPPvkkHRs9FqP7NI/o/IFia/R6ZhdJhJJdJBFKdpFEKNlFEqFkF0mEkl0kEUp2kUSUtM4+atQo2qMc9UazeFSTZcvvAvG2y6zv+9VXX6VjZ82aReNRz3merayjZa6jXvpx48YVfduRqIYf9ZwfO3aMxp966qnMWPR4idZHyPN3R+PZGgAAcOmll2bGPv/88+zbjKclIv8MlOwiiVCyiyRCyS6SCCW7SCKU7CKJULKLJKKkdfa+vj6cOXMmMz5hwgQ6ntV8o7pnVMNnffYA7wuPerqjvuuoxr969WoaZ33bkajW/dhjj9F4dNz37NmTGXv55Zfp2Kive9IkvnEwG8/WXgeA48eP03i0BkHU787OvZgzZw4d293dXdTt6pldJBFKdpFEKNlFEqFkF0mEkl0kEUp2kUQo2UUSMZT92WcC+COASwGcB9Ds7r81sycAPAjg26biR939LXZdVVVVtE84qrOPHz8+MxbVXKP+5Ki/mdWToz3Oo5osO/cAiP+2jo6OzNhXX31Fx/b29tL4448/TuNRvzur+0Zjo+MWHXf2eInObYjOP4h6zmtqamh85syZmbHovA32d7HzQYZyUs05AL9w981mNh7AJjN7pxD7jbs/M4TrEJEyG8r+7G0A2grfd5rZTgCNIz0xERle/9B7djO7HMDVAP5euGiVmW0zs+fNbNDXmma20sxazKzl5MmT+WYrIkUbcrKb2TgAfwLwc3f/CsDvAFwBYDH6n/l/Ndg4d2929yZ3b4ree4rIyBlSspvZKPQn+svu/mcAcPej7t7n7ucB/B7A0pGbpojkFSa79X9M/RyAne7+6wGXD1wm9k4AO4Z/eiIyXIbyafyNAH4MYLuZbSlc9iiAe81sMQAH0ArgJ9EVjR49GgsWLMiMR+UOVv6KSmddXV254u3t7ZmxqIyzdetWGl+3bh2Nd3Z2Fh2PWi2jNtJofNS+y+7TqMQUtc9Gt33q1Kmirzsq1bLyFxBvEc6uPyopsrId+7uG8mn8hwAGyyRaUxeRyqIz6EQSoWQXSYSSXSQRSnaRRCjZRRKhZBdJREmXknZ3WhvNUwtvbW2lY6M20g8++IDGt23blhnbv38/HRu1akY136jWzeJ5WneHIhrPaunRdtE9PT00Hp2Xwc4hiJYWv/zyy2m8oaGBxqNzI9hS1tFjlZ3XwfJLz+wiiVCyiyRCyS6SCCW7SCKU7CKJULKLJELJLpIIi2qVw3pjZscA7Btw0RQAfG/c8qnUuVXqvADNrVjDObdZ7j51sEBJk/17N27W4u5NZZsAUalzq9R5AZpbsUo1N72MF0mEkl0kEeVO9uYy3z5TqXOr1HkBmluxSjK3sr5nF5HSKfczu4iUiJJdJBFlSXYzW25mu8zsCzN7pBxzyGJmrWa23cy2mFlLmefyvJm1m9mOAZfVmdk7Zra78LUse2plzO0JMztUOHZbzGxFmeY208z+ZmY7zexTM/tZ4fKyHjsyr5Ict5K/ZzezagCfA7gVwEEAGwHc6+6flXQiGcysFUCTu5f9BAwzuxlAF4A/uvu/FC77TwAd7r668B/lJHf/jwqZ2xMAusq9jXdht6LpA7cZB3AHgH9HGY8dmde/oQTHrRzP7EsBfOHue9z9GwCvAri9DPOoeO6+DkDHBRffDuDFwvcvov/BUnIZc6sI7t7m7psL33cC+Hab8bIeOzKvkihHsjcCODDg54OorP3eHcBfzWyTma0s92QGMc3d24D+Bw+A+jLP50LhNt6ldME24xVz7IrZ/jyvciT7YIuiVVL970Z3vwbAbQB+Wni5KkMzpG28S2WQbcYrQrHbn+dVjmQ/CGDgznQzABwuwzwG5e6HC1/bAaxB5W1FffTbHXQLX7N3nCyxStrGe7BtxlEBx66c25+XI9k3AphnZrPN7GIA9wB4vQzz+B4zqyl8cAIzqwHwQ1TeVtSvA7i/8P39ANaWcS7fUSnbeGdtM44yH7uyb3/u7iX/B2AF+j+R/xLAY+WYQ8a85gDYWvj3abnnBuAV9L+sO4v+V0QPAJgM4F0Auwtf6ypobi8B2A5gG/oTa3qZ5rYM/W8NtwHYUvi3otzHjsyrJMdNp8uKJEJn0IkkQskukgglu0gilOwiiVCyiyRCyS6SCCW7SCL+HxhMffdsMV66AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from PIL import Image\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n",
    "from torch import nn\n",
    "from torch.autograd import Variable\n",
    "import torch.nn.functional as F\n",
    "\n",
    "# 读入一张灰度图的图片\n",
    "im = Image.open('C:/Users/DELL/image/2.jpg').convert('L') \n",
    "im = np.array(im, dtype='float32') # 将其转换为一个矩阵\n",
    "\n",
    "# 将图片矩阵转化为 pytorch tensor，并适配卷积输入的要求\n",
    "im = torch.from_numpy(im.reshape((1, 1, im.shape[0], im.shape[1]))) \n",
    "\n",
    "#定义一个算子对其进行轮廓检测\n",
    "\n",
    "# 使用 nn.Conv2d\n",
    "'''使用 nn.Conv2d() 会帮我们默认定义一个随机初始化的 weight，如果我们需要修改，\n",
    "那么取出其中的值对其修改，如果不想修改，那么可以直接使用这个默认初始化的值，非常方便\n",
    "实际使用中基本都使用 nn.Conv2d() 这种形式'''\n",
    "conv1 = nn.Conv2d(1, 1, 3, bias=False) # 定义卷积\n",
    "\n",
    "sobel_kernel = np.array([[-1, -1, -1], [-1, 8, -1], [-1, -1, -1]], dtype='float32') # 定义轮廓检测算子\n",
    "sobel_kernel = sobel_kernel.reshape((1, 1, 3, 3)) # 适配卷积的输入输出\n",
    "conv1.weight.data = torch.from_numpy(sobel_kernel) # 给卷积的 kernel 赋值\n",
    "\n",
    "edge1 = conv1(Variable(im)) # 作用在图片上\n",
    "edge1 = edge1.data.squeeze().numpy() # 将输出转换为图片的格式\n",
    "\n",
    "# 池化层\n",
    "'''利用了图片的下采样不变性，即一张图片变小了还是能够看出了这张图片的内容，\n",
    "而使用池化层能够将图片大小降低，非常好地提高了计算效率，\n",
    "同时池化层也没有参数。池化的方式有很多种，比如最大值池化，均值池化等等，\n",
    "在卷积网络中一般使用最大值池化nn.MaxPool2d()。'''\n",
    "# 使用 nn.MaxPool2d\n",
    "pool1 = nn.MaxPool2d(9, 11)\n",
    "print('before max pool, image shape: {} x {}'.format(im.shape[2], im.shape[3]))\n",
    "small_im1 = pool1(Variable(im))\n",
    "small_im1 = small_im1.data.squeeze().numpy()\n",
    "print('after max pool, image shape: {} x {} '.format(small_im1.shape[0], small_im1.shape[1]))\n",
    "\n",
    "plt.imshow(small_im1, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再对所有图片处理一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(30, 784)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[225., 228., 244., ..., 238., 244., 243.],\n",
       "       [241., 137.,  90., ..., 254., 251., 254.],\n",
       "       [ 20.,  32.,  42., ..., 138., 120., 124.],\n",
       "       ...,\n",
       "       [245., 246., 240., ..., 255., 247., 254.],\n",
       "       [254., 255., 255., ..., 174., 182., 176.],\n",
       "       [207., 208., 210., ..., 233., 233., 234.]], dtype=float32)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "picture_array = []\n",
    "for i in range(30):\n",
    "    im = Image.open('C:/Users/DELL/image/%s.jpg'% i).convert('L') # 读入一张灰度图的图片\n",
    "    im = np.array(im, dtype='float32') # 将其转换为一个矩阵\n",
    "    im = torch.from_numpy(im.reshape((1, 1, im.shape[0], im.shape[1]))) \n",
    "    pool1 = nn.MaxPool2d(9, 11)\n",
    "    small_im1 = pool1(Variable(im))\n",
    "    small_im1 = small_im1.data.squeeze().numpy()\n",
    "    small_im1 = small_im1.reshape(-1)\n",
    "    picture_array.extend(small_im1)\n",
    "picture_array=np.array(picture_array).reshape(30,784)\n",
    "print(np.shape(picture_array))\n",
    "picture_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对导入的数据进行处理："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([8, 8, 8, 8, 8, 8, 8, 8, 8, 0, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,\n",
       "       8, 8, 8, 8, 1, 8, 8, 8], dtype=uint8)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img1_pred = mlp.predict(picture_array)\n",
    "img1_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.结果分析\n",
    "  最后的预测结果好像很糟糕，但是也不是很清楚原因到底是什么。我觉得有可能是淘宝上面的照片太高级了，图片上除了物品之外还有很多的文字和数字在上面，然后我的模型又不是很好，样本的图片又有可能太简单了，所以才会产生这样子的结果。感觉弄了那么久结果好像没什么用，有点难受。。。\n",
    "\n",
    "## 4.总结\n",
    "  第三个报告给我的感觉就是好难啊，不像前两个一样结果好像都还可以。这个去预测爬虫爬下来的东西好像完全不行，但是在爬虫的部分之前，还是锻炼了我处理图片类型的数据的能力。总的来说，三个报告都已经完全结束了，所以感觉做下来是真的学到了很多的东西。就像之前敲代码也就是在spyder上面敲一遍，然后删掉进行下个任务。但是这个报告因为要用notebook写，我还专门跑去b站学习了怎么用notebook编写markdown，感觉收获很大，当然看视频的时候那个老师还一直在说html写markdown很厉害，有空一定去学一学。同时对python的这些库也了解的更加深入了，感觉运用起来灵活了很多呢，同时也对机器学习的几个方法有了更为深入的探索。这门课虽然说结束了，但是我们对于机器学习的探索一定还能越走越远。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
